Nature Methods

Deep learning can extract quantitative measurements from microscopy images that are inaccessible to classical analysis, but developing these models requires machine learning expertise that most imaging scientists do not have. Here we present a framework in which a researcher describes their microscopy problem to a large language model (LLM) agent in under ten minutes of conversation--specifying what they image, what they want to measure, and what success looks like--and the agent autonomously handles the rest: designing physics-based training data, implementing a neural network, training, diagnosing failures, and iterating without human intervention. A researcher can start the agent before leaving the lab; overnight, it tests tens to a hundred model variations, each one an experiment that would otherwise demand active attention. We validate the framework across six microscopy modalities and four problem types. On the BBBC039 nuclear segmentation benchmark, the agent autonomously trains a U-Net with 3-class semantic segmentation and morphological post-processing, achieving pixel-level Dice of 0.97 and object-level F1 of 0.84--within 7% of the published baseline--while diagnosing a data pipeline bug that no amount of hyperparameter tuning could resolve. On single-protein holographic microscopy, the agent reads a published paper, designs a simulator, and develops an optimized model in a single session. On PatchCamelyon histopathology classification, the agent autonomously evolves through four optimization phases--from scratch training through transfer learning and regularization to inference-time ensembling--completing 97 iterations on 262,144 images to reach 89.3% test accuracy and 96.3% AUC, nearly matching the published rotation-equivariant baseline. This framework enables microscopy researchers to use deep learning-based image analysis without machine learning domain knowledge.

Advances in tissue clearing and light-sheet microscopy enable cellular resolution whole-brain 3D imaging. However, whole-brain quantification tools do not yet meet demands for efficiency or assess morphometry. Here we present CellPheno, a 3D nuclei instance segmentation framework for high-throughput cellular phenotyping. CellPheno quantifies an entire P4 mouse brain within 15 hours. We showcase whole-brain morphometry, enhanced stitching, and co-localization across multiple cell types in 53 brains.

Computational methods have advanced the analysis of animal behavior, yet significant challenges remain in data standardization, analytical reproducibility, and workflow integration. Existing computational solutions often demand extensive programming proficiency or compel users to navigate a highly fragmented ecosystem of disconnected tools for tracking, statistical analysis, and visualization. Here, we present EthoClaw, an open-source, artificial intelligence-driven workflow platform built upon the OpenClaw agentic framework, functioning as a locally deployable AI assistant for behavioral research. EthoClaw provides an integrated computational infrastructure that seamlessly bridges the gap between raw behavioral video acquisitions and publishable scientific results. In this study, we demonstrate the platforms capacity to natively ingest video data via a dual-mode tracking architecture: utilizing ultra-fast image processing for rapid object detection, and leveraging the SuperAnimal methods for precise, markerless postural tracking. To ensure maximal interoperability, EthoClaw automatically converts various tracking data formats into DeepLabCut-compatible formats, enabling high-throughput phenotyping by generating publication-quality visualizations alongside rigorous multidimensional statistical profiling. Furthermore, the platform incorporates a large language model (LLM)-driven reporting module that dynamically synthesizes analytical documents, ensuring methodological transparency. Through an open field test, we validate the practical usability of EthoClaw while accelerating computational throughput by localizing heavy video processing to circumvent cloud bandwidth bottlenecks. Operating via an omnichannel natural language interface that integrates seamlessly with ubiquitous instant messaging software, EthoClaw democratizes advanced computational behavioral analysis, offering a holistic, highly efficient ecosystem that enforces experimental reproducibility and open science principles.

Three-dimensional branching networks exist throughout biological, natural, and man-made systems as pathways through volumetric space. Segmentation is required to correctly reconstruct the networks in whole or in part for analysis. This presents a unique challenge as minor voxel misclassifications can cause sporadic connectivity shifts, whereby connected elements appear to disconnect (false negatives) or to even become amplified (false positives). Addressing this topological vulnerability requires the generation of 3D models since 2D slice-by-slice approaches cannot maintain connectivity across x, y, and z axes. Yet tracking 3D architecture demands substantially more analytical resources than using a 2D strategy as generating volumetric annotations requires extraordinary amounts of expert time to manually annotate. This creates a fundamental annotation bottleneck: with sparse training data available, deep learning models tend to overfit available volumes and fail to generalize to novel volumes. We present a cascade training workflow that overcomes this bottleneck through a positive feedback loop in which trained models become annotation aids for subsequent volumes. The workflow begins with random forests that generate initial drafts from minimal labels, followed by expert refinement that cycle ever closer to the ground truth. As refined data accumulates, training transitions from 2D to 3D architectures, which systematically expand sparse datasets into comprehensive training sets. The outcome is a 3D nnU-Net model optimized for topology-preserving segmentation. We dub our resulting model DeepBranchAI. Training validation on heavily branching mitochondrial networks, generated by focused ion beam scanning electron microscopy (FIB-SEM, 15nm voxel resolution) achieved Dice Similarity Coefficient (DSC) = 0.942 across 5-fold cross-validation. Transfer learning to vascular networks (VESSEL12 dataset, CT volumes, 30,000-fold voxel size difference) training on as little as 10% of target data achieved 97.05% accuracy against ground truth, validating that learned features represent domain-general topological principles. This workflow reduces annotation time from months to weeks while transforming sparse initial labels into robust training sets. Complete implementation, trained weights, and validation code are provided open source.

High-plex spectral imaging has the potential to transform the analysis of spatial organization in cells and tissues, yet its practical implementation remains limited by challenges in panel design, sample preparation, signal balancing, and experimental validation. While cyclic imaging approaches are widely used in spatial omics, spectral imaging across the full fluorescence spectrum and computational unmixing remain underutilized due to these challenges. Here, we present a generalizable framework for high-plex spectral imaging that leverages DNA-barcoded labeling and programmable signal amplification to provide precise control over fluorescence signal composition. Orthogonal DNA barcodes decouple target labeling from fluorophore detection, enabling reversible fluorophore application and systematic panel optimization directly on the same sample. Programmable DNA-based amplification further enables independent and quantitative tuning of fluorescence intensities across targets, overcoming a key limitation of spectral unmixing, namely imbalanced signal contributions in overlapping channels, and thereby improving accuracy and robustness. The framework also supports the generation of experiment-specific ground truth datasets and systematic evaluation of unmixing algorithms, providing a quantitative basis for panel validation and performance assessment. We demonstrate the practical implementation of this framework by developing a panel for simultaneous imaging of 15 subcellular structures without fluidic cycling and using the optimized panel to profile the effects of chemical perturbations on subcellular organization. We quantitatively evaluate panel compilation and provide a rigorous assessment unmixing performance using both linear and reference-free unmixing methods. Importantly, we leverage foundation models trained on standard fluorescence data, for segmentation-free, high-dimensional analysis of spectrally unmixed images without needing large datasets or model retraining. Together, we establish a practical and tunable framework for high-plex spectral imaging that lowers experimental barriers and enables broader adoption of spectral unmixing for biological and biomedical applications.

Spatial transcriptomics enables measurement of gene expression while preserving spatial organization within tissues. Accurate reconstruction of single-cell transcriptomes requires precise whole-cell segmentation, yet many spatial transcriptomics experiments provide only nuclear staining images, making reliable inference of cell boundaries difficult. Here we introduce Halo, a pretrained segmentation model that reconstructs whole-cell boundaries by integrating nuclear morphology with the spatial distribution of RNA transcripts. Halo converts transcript coordinates into molecular density maps that are processed jointly with DAPI images using a Cellpose-SAM segmentation architecture. Unlike existing approaches that require dataset-specific training, Halo is pretrained on multimodal Xenium data from 12 tissue types and can be directly applied to new datasets without additional training. Across diverse tissues, Halo substantially outperforms the widely used nuclear expansion strategy, achieving higher agreement with ground-truth cell boundaries and more accurate RNA-to-cell assignment. Improved segmentation leads to more reliable cell type identification and more accurate estimation of cell morphological features. By providing a pretrained, generalizable model for whole-cell reconstruction, Halo enables scalable and reproducible cell segmentation for image-based spatial transcriptomics.

Automating cellular organelle segmentation is key to increasing the throughput in electron microscopy (EM) and volume EM (vEM) workflows. Deep learning (DL) has accelerated this process, but model development has predominately centered on mitochondria, partly because of a scarcity of suitable training datasets for other features. Here, we crowdsourced the manual step of labeling nuclei and lipid droplets (LDs) from complex cellular EM images and trained Panoptic DeepLab (PDL) models on these large, heterogenous annotated datasets as well as on publicly available vEM datasets. NucleoNet and DropNet, the resulting instance segmentation models for nuclei and LDs, respectively, yield high-quality results on varied benchmarks. We applied these models to quantify differences between 2D and 3D in vitro cancer models versus in vivo tumors, highlighting a path toward robust quantitation in EM. NucleoNet and DropNet are publicly available on our napari plugin, empanada v1.2, for easy point-and-click segmentation of 2D and 3D cellular EM images.

Spatial transcriptomics (ST) technologies enable measurement of gene expression with spatial context, offering unprecedented insight into tissue architecture and cellular microenvironments. A fundamental analysis task is the identification of spatial domains, i.e., contiguous regions with distinct molecular profiles. As ST datasets scale to larger tissue areas, multiple slices, and multiple molecular modalities, there is a growing need for clustering methods that are accurate, scalable, and capable of integrating diverse spatial and molecular signals. We present GraphBG, a unified and scalable framework for spatial domain detection in ST data. GraphBG integrates approximate spectral graph convolutions with a variational Bayesian Gaussian mixture model, enabling robust representation learning and clustering of spatially coherent domains. We extend this core model to support multi-slice analysis (GraphBG-MS) through metacell aggregation, batch correction, and joint clustering, and to multi-modal spatial omics data (GraphBG-MM) via modality-specific graph encodings and kernel canonical correlation analysis. Across diverse real and simulated datasets, GraphBG consistently outperforms existing methods in domain coherence, scalability, and biological interpretability. Notably, it accurately clusters over 370,000 cells from 31 MERFISH tissue slices in just 5 minutes and integrates spatial transcriptomic and proteomic data for improved domain resolution. Applying GraphBG-MS to mouse liver ST data, we show that it captures canonical lobular zonation and disease-specific remodeling, highlighting its ability to reveal biologically meaningful tissue organization.

Genetically encoded voltage indicators (GEVIs) enable cell-type-specific optical readout of membrane potential, but two-photon (2P) spike detection has been hampered by low signal-to-noise and ultrafast off-kinetics, restricting use to specialized microscopes. We introduce FORCE1s, a green, positive-going GEVI engineered to make robust 2P voltage imaging broadly accessible. FORCE1s brightens from a dark baseline during depolarization, reports spikes with [~]100% {Delta}F/F in awake mice, and displays repolarization kinetics that are tuned for reliable spike detection at sub-kilohertz frame rates. As a result, FORCE1s supports spike-resolved multi-cell recordings on standard resonant-scanning microscopes, and further scales to larger fields of view and neuron counts on advanced modalities. FORCE1s also enables multiplexed voltage-neurotransmitter imaging and extended recordings in freely moving mice using a compact, affordable MEMS-based 2P miniscope. Together, these advances establish FORCE1s as a community-ready tool that democratizes deep-tissue voltage imaging across platforms and experimental contexts.

We apply TeaCache, an adaptive caching technique from video diffusion to SimpleFolds flow-matching protein structure prediction and achieve (9 to 14)-fold inference speedups with negligible quality loss. We determine that flow matchings near-linear generative trajectories make consecutive neural-network evaluations highly redundant. At a low redundancy threshold, SimpleFold-Turbo (SF-T) skips {approx} 93 % of forward passes while preserving near-baseline template modeling (TM)-scores across 300 structurally diverse CATH domains and all six SimpleFold model sizes (100 million to 3 billion parameters), at compute budgets where log-uniform step-skipping collapses. Speedup scales with model size because caching overhead is constant while per-step cost grows, and a general three-phase skip pattern emerges independent of protein size or fold. SF-T requires no retraining, no weight modification, and no MSA server dependencies. We release SF-T as fully open-source software enabling thousands of structure predictions per hour on commodity hardware.

Upon T cell receptor (TCR) engagement, a T cell forms an immunological synapse (IS) with an antigen-presenting cell (APC), which can be mimicked by purified ligands on supported lipid bilayers (SLBs)1,2. Microvilli actively scan the surface; upon initial engagement, F-actin-dependent TCR microclusters form, and the central supramolecular activation cluster (cSMAC) sustains TCR signaling in CD8 T cells3,4. Although signaling activities within the IS have been observed qualitatively through total internal reflection immunofluorescence microscopy5-7, the stoichiometric relationships among the components of the TCR signalosome remain unknown. In this study, we employed a two-step approach to quantify the components of the TCR signalosome. First, Jurkat cell lines expressing GFP-tagged proteins on a knockout background were used to calibrate fluorescence intensity (IF) signals against molecular copy numbers, based on measurements of single-tag signals and multiple corrections. In the second step, this calibration was applied to determine the stoichiometries of key TCR signalosome components, including TCR, CD8, CD28, CD45, PD-1, Lck, ZAP-70, LAT, and PLC{gamma}1, across scanning, early activation, and sustained activation states in human primary T cells. We refer to the method as quantitative extrapolation from single-tags (QuEST) immunofluorescence microscopy. Applying the QuEST, we were surprised to find that the ZAP-70:TCR ratio in microclusters and the cSMAC was 1:1, far from the potential 10:1 ratio. Nanoscale structures of the TCR signalosome were further captured using direct stochastic optical reconstruction microscopy (dSTORM), confirming that ZAP-70 was strongly co-localized with the TCR. Moreover, we applied QuEST to confirm the presence of T cell intrinsic CD28 recruitment, independent of CD80 or CD86 on SLBs, during TCR activation. This T cell intrinsic CD28 recruitment could be disrupted through engagement of PD-1 with PD-L1 on SLBs. This shows that PD-1 engagement can disrupt T cell intrinsic CD28 costimulation. QuEST provides a broadly applicable pipeline for quantitative analysis of TCR signalosomes in human primary cells, enabling a quantitative basis for the rational manipulation and engineering of the TCR signalosome in immunotherapies.

Reconstructing natural images from brain activity represents one of the most compelling demonstrations of the synergy between modern neuroimaging and machine learning. However, the computational pipelines underlying these results remain scarcely accessible, difficult to reproduce, and offer limited opportunities for hands-on experimentation. They depend on large codebases, expensive hardware, and multiple representational stages whose interactions are not obvious. We present a step-by-step tutorial, organized across six notebooks, for reconstructing natural images from fMRI responses in the Natural Scenes Dataset. The workflow walks the reader through three main stages: predicting coarse image structure from brain activity by targeting the latent space of a pretrained image autoencoder, predicting semantic content by targeting learned vision-language embeddings, and combining both signals through a pretrained generative model that produces a final image reflecting both the recovered layout and the recovered meaning. Each notebook explains the reasoning behind its pipeline stage and provides runnable code to reproduce and build on each component. We present qualitative and quantitative metrics for all of our pipeline stages. Every notebook runs end-to-end on free-tier Google Colab hardware, and each stage can be inspected, modified, and replaced independently.

Multiplexed spatial proteomics and spatial transcriptomics generate large, high-dimensional imaging datasets that are challenging to visualize efficiently, particularly at whole-slide and cohort scale. Visualization is an essential step for rapid detection of staining artefacts, such as protein aggregates or non-specific staining. Here, we present Odon, a native Rust desktop viewer designed for rapid, interactive exploration of multiplex imaging data on a standard laptop. Odon is primarily built around the OME-Zarr imaging format, and supports annotations via GeoJSON and GeoParquet, with secondary support for SpatialData, Xenium containers, and TIFF. Data can be stored locally or streamed directly from HTTP or S3-compatible object storage using viewport-driven tile loading. Odon incorporates a highly optimized rendering engine that substantially outperforms existing viewers. In benchmarking, Odon loaded a 32 GB, 36-plex whole-slide OME-Zarr image in under 1 second, compared with 10.14 seconds for QuPath and 35 seconds for Napari. Its GPU-based compositing pipeline also enables smooth rendering and interaction with more than 1,000,000 segmented cells, exceeding the practical limits of many existing tools. Odon further supports integrated visual analytics, including live thresholding and cell selection, and a mosaic mode for simultaneous viewing of hundreds of regions of interest in cohort and tissue microarray studies. Together, these features establish Odon as a high-performance platform for scalable visualization of spatial proteomics data.

Accurate genome annotation is fundamental to modern biology, yet distinguishing authentic protein-coding sequences from prediction artifacts remains challenging, particularly in complex plant genomes where automated methods are error-prone and manual curation is rarely feasible due to prohibitive time and costs. Here, we present GAP-MS (Gene model Assessment using Peptides from Mass Spectrometry), an automated proteogenomic pipeline that leverages mass spectrometry evidence to systematically validate the protein-level accuracy of predicted gene models. Applied across 9 major crop species, GAP-MS consistently improved prediction precision for four widely used gene prediction tools. In addition to filtering erroneous models, the pipeline identified hundreds of previously missing gene models from current standard reference annotations. These peptide-supported loci were further verified by transcriptional evidence, well-supported functional annotations, and high coding-potential scores. Together, these results demonstrate that direct proteomic evidence provides a robust framework for resolving annotation ambiguities, defining high-confidence reference proteomes, and uncovering overlooked protein-coding genes, while facilitating the identification of sequences that may require further investigation. GAP-MS is freely available as a web interface at https://webclu.bio.wzw.tum.de/gapms/.

The spatial organization of proteins within cells governs virtually all cellular functions. Yet, current imaging technologies can simultaneously visualize only tens of proteins, orders of magnitude below the thousands that populate a single human cell. Here, we present ProtiCelli, a deep generative model that simulates microscopy images for 12,800 human proteins from just three cellular landmark stains. Trained on 1.23 million images from the Human Protein Atlas, ProtiCelli outperforms existing methods in reconstruction accuracy and textural fidelity, and generalizes to unseen cell types and drug perturbations absent from training. We demonstrate that ProtiCelli-generated images preserve hierarchical subcellular organization, recapitulate known protein-protein interaction landscapes, and resolve compartment-specific functions of moonlighting proteins at the single-cell level. Remarkably, the model infers drug-induced changes in protein expression and localization from cell morphology alone, predicts cell cycle stage without dedicated cell cycle markers, and enables unsupervised segmentation of subcellular compartments as well as spatial decomposition of gene sets into functional regions. Ultimately, we leverage ProtiCelli to generate Proteome2Cell, an unprecedented dataset of 30.7 million simulated images creating 2,400 "virtual cells" across 12 human cell lines. These proteome-scale images enable the construction of hierarchical single-cell models that distinguish conserved from dynamic protein architectures. Integration of Proteome2Cell into the Human Protein Atlas democratizes the exploration of these "virtual cells". By computationally bridging the experimental scalability gap, ProtiCelli establishes a foundation for spatial virtual cell modeling and paves an avenue for transforming spatial proteomics from cataloging proteins to simulating complete cellular systems.

We present mdBIRCH, an online clustering method that adapts the BIRCH CF-tree to molecular dynamics (MD) data by using a merge test calibrated directly to RMSD. Each arriving frame is routed to the nearest centroid and added only if the post-merge radius computed from the cluster feature remains within a user-supplied threshold. This keeps the average deviation to each cluster centroid bounded as the cluster grows and preserves a simple interpretation of resolution in physical units. We evaluate mdBIRCH on a {beta}-heptapeptide and the HP35 system. We propose two protocols to make the threshold selection easier: (a) RMSD-anchored runs that use controlled structural edits to define interpretable operating points and (b) blind sweep that tracks how cluster count, occupancy, and coverage change with the threshold. In both systems, increasing the threshold reduces the number of clusters, concentrates coverage in high-occupancy states, and broadens within-cluster RMSD distributions. Furthermore, because decisions rely only on cluster summaries, mdBIRCH completely avoids the need for pairwise distance matrices, scales near-linearly with the number of frames on standard hardware, and naturally supports incremental operation. The method offers a practical combination of speed and interpretability for large-scale trajectory analysis.

We present a multimodal pipeline for 3D imaging of cerebral white-matter archi-tecture across scales, from whole-brain axonal projections down to individual myelinated axons. After diffusion MRI, an adult ex vivo human hemisphere undergoes label-free imaging with Hierarchical Phase-Contrast Tomography (HiP-CT) from 20 {micro}m/voxel in the whole hemisphere to 2 {micro}m/voxel in areas of interest, with intrinsic cross-scale alignment. A 4 cm tissue block extracted from the hemisphere is reimaged with HiP-CT at 0.857 {micro}m/voxel, enabling direct visualisation of single myelinated axons. After osmium staining, micro-CT at 0.364 {micro}m/voxel and electron microscopy at 4 nm/voxel are acquired in biop-sies from the tissue block to validate the presence of myelinated axons in the label-free HiP-CT contrast. Spanning three orders of magnitude in resolution, these co-registered multimodal datasets bridge microscopic wiring and macro-scopic brain organisation, providing a foundation for anatomically grounded whole-brain connectomics.

To understand how brains work, it is necessary to connect neural activity to synaptic-resolution circuit architecture. Recent advances in light-sheet microscopy (LSM) enable whole-brain, cellular-resolution imaging of activity of all neuronal cell bodies, however, most neurons from such datasets cannot be identified. In most organisms, neurons are identifiable based on their projections (and not based on their cell body position) which, when densely labelled, cannot be resolved using LSM. Here, we present a novel methodology to overcome this by combining whole-brain activity imaging with subsequent volume electron microscopy imaging of the same brain to visualise neuronal projections and identify neurons with interesting activity. We used this approach to identify brain neurons involved in nociception in Drosophila larva. After whole-brain imaging of activity during nociceptive stimulation, we imaged the same brain with an enhanced focused ion-beam electron microscope (eFIB-SEM). We registered the functional and anatomical volumes and reconstructed (in the eFIB-SEM volume) the projections of neurons that responded to nociceptive stimulation to determine their developmental lineage and identity. This revealed a distributed nociceptive network spanning 25 distinct lineages and many distinct brain areas, and included direct brain targets of nociceptive projection neurons that integrate nociceptive information with other sensory modalities, as well as brain output neurons (descending neurons [DN]) that likely contribute to action-selection. Our workflow provides a powerful framework for mapping neuronal activity onto structure across an entire brain, yielding novel insights into the distributed central processing of noxious stimuli.

Single-cell RNA sequencing and spatial transcriptomics have transformed our understanding of the transcriptional landscape by enabling high-resolution profiling of gene expression. Yet most experimental pipelines and their associated analysis frameworks collapse transcript diversity into gene-level counts, obscuring alternative splicing and isoform usage. The increasing ability of long-read sequencing to recover full-length transcripts from single cells and spatially barcoded tissues has created a pressing need for computational frameworks to support the storage, analysis, and visualisation of isoform-resolved data. Existing tools for isoform and splicing analysis either specialise in bulk, single-cell, or spatial RNA-seq assays in isolation and remain fragmented across languages and data models, limiting interoperability and hindering widespread adoption. We present Allos, a Python framework for isoform-level single-cell and spatial transcriptomics analysis. Built on the AnnData data model, Allos natively represents transcript-level quantification and integrates directly with GTF/GFF and FASTA annotations. Allos enables differential isoform usage screening, multi-panel visualisation, structural transcript interpretation, and protein-level analysis across bulk, single-cell, and spatial assays from both long- and short-read sequencing. Its modular design and scverse compatibility allow isoform-resolved analyses to run alongside established gene-level workflows, linking transcript-level screening with structure-aware visualisation and downstream interpretation. Allos is open-source and available at https://github.com/cobioda/allos, with comprehensive documentation and tutorials provided online.

The cerebral vasculature is central to brain function, with alterations linked to numerous cerebrovascular and neurological disorders. Yet, no single imaging modality can capture the entire cerebral vascular network in humans. Instead, an array of techniques are sensitized to different spatial scales, while trading off resolution for coverage. Magnetic Resonance Imaging (MRI) typically resolves only large pial vessels, while high-resolution microscopy allows micrometer-scale vessels to be mapped over limited spatial extents. These techniques must therefore be combined to obtain a complete mapping of the cerebral angioarchitecture, which underscores the need for automatic, cross-modal vessel segmentation. Here, we introduce VesSynth, a flexible vessel segmentation framework that achieves state-of-the-art accuracy across multiple modalities and spatial resolutions (MR, optical and X-ray imaging), despite being trained entirely on synthetic data. By enabling consistent vascular mapping across scales, this framework paves the way to comprehensive investigation of cerebrovascular organization and its role in health and disease.